Upcoming Event: Oden Institute & Dept. of Neuroscience

Theory of Sample Efficient Learning in Neural Population Codes

Blake Bordelon, Ph.D. Program, Harvard University

Abstract

Learning from a limited amount of experience requires a suitable inductive bias since many general rules are consistent with a finite number of observations. To study how and when sample-efficient generalization is possible, I will develop an analytically tractable theory of sample-efficient learning which relates structure of a neural representation to its functional biases through a spectral decomposition of the population code. This theory predicts the generalization error of a learned readout from the code as a function of the amount of data, while also providing a novel interpretation of similarity analysis. I will discuss applications of this theory to visual learning tasks using mouse visual cortex data, and provide analysis of how representations of space can influence the speed of temporal difference learning in navigation tasks. This theory also predicts generalization of deep neural networks operating in the "kernel" or "lazy learning" regime. To go beyond the static population code theory (kernel regime), I will describe a novel mean field theory for randomly initialized multi-layer networks which allow adaptation of the hidden representations during learning. This mean field theory illustrates how the learning rule, multi-layer architecture, and structure of the task alter the dynamics of learning and the learned representations in hidden layers of the network.

Biography

Blake Bordelon is an Applied Math PhD student at Harvard University researching theoretical neuroscience and deep learning. His work often employs mathematical techniques from physics, dynamical systems, and random matrix theory to analyze the learning dynamics of large neural networks. Before his PhD, he studied engineering and physics at Washington University in St. Louis for his undergraduate work where he performed research in many-body quantum mechanics and computational neuroscience.